Display control device, parking assist apparatus and display control method

ABSTRACT

Provided is a display control apparatus that generates a display overhead image including overhead images of each of a vacant parking space and a vehicle and outputs the display overhead image to a display apparatus. The display overhead image includes at least one of an overhead image of the vacant parking space outside of a current viewing angle of a vehicle-mounted camera and an overhead image of the vacant parking space hidden behind a static object as seen from a current position of the vehicle.

TECHNICAL FIELD

The present disclosure relates to a display control apparatus, a parkingassistance apparatus, and a display control method.

BACKGROUND ART

For example, the related art discloses a parking assistance apparatusthat converts an image of a vacant parking space captured by a camerafor capturing the periphery of a vehicle to an overhead image, generatesa display image by combining an image indicating this vehicle withrespect to the overhead image, and displays the display image.

CITATION LIST Patent Literature PTL 1 Japanese Patent ApplicationLaid-Open No. 2009-083735 Solution to Problem

A display control apparatus according to an aspect of the presentdisclosure includes: a receiver that, for each detection of a vacantparking space present in a periphery of a vehicle, receives locationalinformation on the vacant parking space and a captured image of thevacant parking space captured by a camera mounted on the vehicle andstores, in a storage, the locational information on the vacant parkingspace and the captured image of the vacant parking space in associationwith each other; and a generator that generates an overhead image of thevacant parking space based on a captured image of the vacant parkingspace captured at a position where the vehicle has been positionedbefore a current point in time, generates a display overhead imageincluding at least the overhead image of the vacant parking space andthe overhead image of the vehicle based on positional information oneach of the vacant parking space and the vehicle, and outputs thedisplay overhead image to a display apparatus mounted on the vehicle,wherein the display overhead image to be displayed on the displayapparatus includes at least one of an overhead image of the vacantparking space outside of a current viewing angle of the camera and anoverhead image of the vacant parking space hidden behind a static objectas seen from a current position of the vehicle.

A parking assistance apparatus according to an aspect of the presentdisclosure includes: a detector that detects a vacant parking spacepresent in a periphery of a vehicle; a traveling controller thatcontrols traveling of the vehicle and automatically parks the vehicle inthe vacant parking space; a receiver that, for each detection of thevacant parking space by the detector, receives locational information onthe vacant parking space and a captured image of the vacant parkingspace captured by a camera mounted on the vehicle and stores thelocational information on the vacant parking space and the capturedimage of the vacant parking space in association with each other; and agenerator that generates an overhead image of the vacant parking spacebased on a captured image of the vacant parking space captured at aposition where the vehicle has been positioned before a current point intime, generates a display overhead image including at least the overheadimage of the vacant parking space and the overhead image of the vehiclebased on positional information on each of the vacant parking space andthe vehicle, and outputs the display overhead image to a displayapparatus mounted on the vehicle, wherein the display overhead image tobe displayed on the display apparatus includes at least one of anoverhead image of the vacant parking space outside of a current viewingangle of the camera and an overhead image of the vacant parking spacehidden behind a static object as seen from a current position of thevehicle.

A display control method according to an aspect of the presentdisclosure is performed by an apparatus mounted on a vehicle, thedisplay control method including: receiving, for each detection of avacant parking space present in a periphery of the vehicle, locationalinformation on the vacant parking space and a captured image of thevacant parking space captured by a camera mounted on the vehicle andstoring the locational information on the vacant parking space and thecaptured image of the vacant parking space in association with eachother; and generating an overhead image of the vacant parking spacebased on a captured image of the vacant parking space captured at aposition where the vehicle has been positioned before a current point intime, generating a display overhead image including at least theoverhead image of the vacant parking space and the overhead image of thevehicle based on positional information on each of the vacant parkingspace and the vehicle, and outputting the display overhead image to adisplay apparatus mounted on the vehicle, wherein the display overheadimage to be displayed on the display apparatus includes at least one ofan overhead image of the vacant parking space outside of a currentviewing angle of the camera and an overhead image of the vacant parkingspace hidden behind a static object as seen from a current position ofthe vehicle.

Advantageous Effects of Invention

According to the present disclosure, an occupant of a vehicle can easilyrecognize a location of a vacant parking space.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic view of an exemplary positional relationship of avehicle, a parked vehicle, and a vacant parking space, which is providedfor describing findings leading the present disclosure;

FIG. 2 is a block diagram illustrating an exemplary configuration of aparking assistance apparatus according to an embodiment of the presentdisclosure;

FIG. 3 is a flowchart illustrating an exemplary operation of a displaycontrol apparatus according to the embodiment of the present disclosure;

FIG. 4 is a schematic view of an exemplary overhead image according tothe embodiment of the present disclosure;

FIG. 5 is a flowchart illustrating an exemplary operation of a displaycontrol apparatus according to Variation 1 of the present disclosure;

FIG. 6 is a schematic view of an exemplary overhead image according toVariation 1 of the present disclosure; and

FIG. 7 illustrates an exemplary hardware configuration of a computerincluded in the parking assistance apparatus according to an embodimentof the present disclosure.

DESCRIPTION OF EMBODIMENTS

Findings leading to the present disclosure will be described withreference to FIG. 1 . FIG. 1 is a schematic view of an exemplarypositional relationship of a vehicle, parked vehicles, and vacantparking spaces. FIG. 1 illustrates each of the vehicles, parkedvehicles, and vacant parking spaces, in a state as seen from directlyabove. In the present description, a “parked vehicle” refers to avehicle that has been parked and that is other than a vehicle which isto be parked (hereinafter, referred to as a “subject vehicle”). Inaddition, a “vacant parking space” refers to a parking space where noother vehicle is parked.

Parked vehicles V1 and V2 illustrated in FIG. 1 are vehicles other thana subject vehicle and parked in parallel in parking spaces delimited bydivision line L (here, the space where one vehicle can park inparallel). Moreover, each of parking spaces BS1 and BS2 illustrated inFIG. 1 is a vacant parking space.

Vehicle 1 illustrated in FIG. 1 is, for example, a vehicle that performsautomatic parking (e.g., a passenger vehicle).

In the present description, an “automatic parking” may be an automaticparking performed without requiring any driving operations performed byan occupant of the vehicle or may be an automatic parking performedwithout requiring some driving operations by the occupant of thevehicle. The former is, for example, an operation in which a parkingassistance apparatus mounted on the vehicle guides and parks the vehiclein a vacant parking space by controlling acceleration, deceleration,braking, and steering of the vehicle (i.e., by complete automaticdriving) without requiring operations for acceleration, deceleration,braking, and steering. The latter is, for example, an operation in whichthe parking assistance apparatus mounted on the vehicle guides and parksthe vehicle in a vacant parking space by controlling the steeringwithout requiring the operation for steering and by controlling theacceleration, deceleration and braking based on the operation of theoccupant of the vehicle.

Here, as an example, a vacant parking space is assumed to be detectedbased on an image captured by a vehicle-mounted camera mounted onvehicle 1 (hereinafter, referred to as a “captured image”). Note that,since a publicly known method can be used for detecting the vacantparking space based on the captured image, a description thereof will beomitted here.

As illustrated in FIG. 1 , vehicle 1 includes left side camera 3 a forcapturing a left side of vehicle 1, right side camera 3 b for capturinga right side of vehicle 1, front camera 3 c for capturing a front sideof vehicle 1, and rear camera 3 d for capturing the rear of vehicle 1.Incidentally, FIG. 1 illustrates an exemplary case where a single cameraunit is provided for each camera; however, a plurality of camera unitsmay be provided for each camera.

For example, assuming that vehicle 1 at the position illustrated in FIG.1 moves slowly to traveling direction A while capturing with each cameraso as to search for a vacant parking space, and as a result, parkingspaces BS1 and BS2 are detected as vacant parking spaces and parkedvehicles V1 and V2 are detected as static objects based on capturedimages.

Additionally, for example, in a case where an occupant of vehicle 1performs an operation instructing displaying of an overhead image, aconventional apparatus (e.g., the apparatus of PTL 1) converts thecaptured image to the overhead image and combines an image of vehicle 1(e.g., an image of vehicle 1 in a state as seen from directly above)with respect to the overhead image to display.

Here, for example, from among vacant parking spaces BS1 and BS2 detectedas parking spaces, in a case where parking space BS1 is outside of aviewing angle of left side camera 3 a, an overhead image to be displayeddoes not include an image indicating parking space BS1. As a result, theoccupant of vehicle 1 cannot recognize a location of parking space BS1detected as the vacant parking space even when seeing the displayedoverhead image.

An object of the present disclosure is to enable the occupant of thevehicle to easily recognize the location of the detected vacant parkingspace.

The findings leading to the present disclosure has been described above.

Hereinafter, an embodiment of the present disclosure will be describedwith reference to the accompanying drawings. Note that, in all thedrawings, an element the same as a precedent element is given the samereference numeral, and the description thereof may be omitted.

A configuration of parking assistance apparatus 100 according to thepresent embodiment will be described with reference to FIG. 2 . FIG. 2is a block diagram illustrating an exemplary configuration of parkingassistance apparatus 100 according to the present embodiment.

Parking assistance apparatus 100 illustrated in FIG. 2 is mounted on,for example, vehicle 1 illustrated in FIG. 1 . Moreover, parkingassistance apparatus 100 is electrically connected to camera group 3,subject-vehicle position sensor 4, display apparatus 5 and actuatorgroup 6, which are mounted on vehicle 1.

Camera group 3 includes, for example, left side camera 3 a, right sidecamera 3 b, front camera 3 c, and rear camera 3 d, which are illustratedin FIG. 1 . In the following, the cameras included in camera group 3 arereferred to as “each camera.” Each camera outputs the captured imagesobtained by capturing to parking assistance apparatus 100.

Capturing processing and captured image output processing by each cameraare performed repeatedly during a search for a vacant parking space(e.g., while vehicle 1 is traveling along traveling direction A in FIG.1 ).

Subject-vehicle position sensor 4 detects a position and orientation ofvehicle 1 based on, for example, signals received from GlobalPositioning System (GPS) satellites (not illustrated) and outputs thesubject vehicle position information indicating the detected positionand orientation of vehicle 1 to parking assistance apparatus 100.

The above processing by subject-vehicle position sensor 4 is performedrepeatedly during the search for the vacant parking space (e.g., whilevehicle 1 is traveling along traveling direction A in FIG. 1 ).

Note that, in the above embodiment, a description will be given with anexample of detecting the position and orientation of vehicle 1 usingsubject-vehicle position sensor 4, the position and orientation ofvehicle 1 may be detected using a publicly known measure and methodother than subject-vehicle position sensor 4.

Furthermore, although FIG. 2 illustrates only subject-vehicle positionsensor 4, vehicle 1 is equipped with, for example, various sensors suchas an accelerator position sensor detecting a position of an acceleratorpedal, a shift position sensor detecting a position of a shift lever, asteering angle sensor detecting a steering angle, and a wheel speedsensor detecting the rotational speed of each wheel of vehicle 1. Eachof these sensors is electrically connected to parking assistanceapparatus 100 and outputs a signal indicating a detection result toparking assistance apparatus 100. These signals are used when parkingassistance apparatus 100 automatically parks vehicle 1.

Display apparatus 5 is provided in the interior of vehicle 1 anddisplays the overhead images (described in detail later) indicatingvehicle 1 and the periphery of vehicle 1.

Actuator group 6 is an actuator group that executes acceleration,deceleration, braking, steering, and the like of vehicle 1. Actuatorgroup 6 includes, for example, various actuators such as a motoractuator executing acceleration and deceleration, a brake actuatorexecuting braking, and a steering actuator executing steering.

Parking assistance apparatus 100 is an apparatus that detects a vacantparking space and automatically parks vehicle 1 in the vacant parkingspace.

As illustrated in FIG. 7 , parking assistance apparatus 100 includes ashardware, for example, Central Processing Unit (CPU) 501, Read OnlyMemory (ROM) 502 storing a computer program, and Random Access Memory(RAM) 503. CPU 501, ROM 502, and RAM 503 are connected to each other.

Functions of the respective parts of parking assistance apparatus 100and display control apparatus 200 (see FIG. 2 ) described below arerealized by executing the computer program read by CPU 501 from ROM 502.Moreover, the computer program may be stored in a predeterminedrecording medium and provided to a user or the like.

Parking assistance apparatus 100 includes detector 11, travelingcontroller 12, and display control apparatus 200.

Detector 11 detects positions of each vacant parking space and eachstatic object based on captured images received from each camera. Thestatic object may be, but is not limited to, a parked vehicle, ashopping cart, a cone, or the like that is present in a parking space.Incidentally, since a publicly known method can be used for detectingthe positions of each vacant parking space and each static object basedon the captured image, a description thereof will be omitted here.

The above processing by detector 11 is performed repeatedly during thesearch for the vacant parking space (e.g., while vehicle 1 is travelingalong traveling direction A in FIG. 1 ).

Traveling controller 12 controls actuator group 6 such that vehicle 1automatically parks in the vacant parking space detected by detector 11.This control causes vehicle 1 to automatically park in the vacantparking space. Note that, the vacant parking space in which automaticparking is performed may be determined by an operation of the occupantof vehicle 1 or may be determined by parking assistance apparatus 100without the operation of the occupant of vehicle 1.

Next, display control apparatus 200 will be described.

Display control apparatus 200 is an apparatus that controls displayingof an image on display apparatus 5. Specifically, display controlapparatus 200 generates an overhead image (in particular, any one of thefirst to third overhead images to be described later) indicating vehicle1 and its periphery and outputs the overhead image to display apparatus5.

As illustrated in FIG. 2 , display control apparatus 200 includesreceiver 14, storage 15, determiner 16, and generator 17.

Receiver 14 receives locational information on the vacant parking spacedetected by detector 11 and an image of the vacant parking spacecaptured by at least one of each camera (hereinafter, referred to as a“captured image of the vacant parking space”) to store them in storage15 in association with each other.

In addition, receiver 14 receives the position information on the staticobject detected by detector 11 and an image of the static objectcaptured by at least one of each camera (hereinafter, referred to as a“captured image of the static object”) to store them in storage 15 inassociation with each other.

Storage 15, under the control of receiver 14, stores the locationalinformation on the vacant parking space and the captured image of thevacant parking space in association with each other.

Storage 15, under the control of receiver 14, also stores the positionalinformation on the static object and the captured image of the staticobject in association with each other.

Moreover, storage 15 stores a prepared subject-vehicle image. Thesubject-vehicle image is an overhead image of vehicle 1.

Furthermore, storage 15 stores a prepared notification image. Thenotification image is an image for notifying the occupant of vehicle 1that the vacant parking space is detected. For example, the notificationimage is an image indicating the detection of the vacant parking space,using characters or illustrations.

Note that, although FIG. 2 illustrates an exemplary case where storage15 is provided in display control apparatus 200, storage 15 may beprovided outside display control apparatus 200 or outside parkingassistance apparatus 100.

Determiner 16 performs various kinds of determination processing basedon the detection result of detector 11. The determination processingwill be described in detail later.

Generator 17 generates an overhead image based on images read fromstorage 15 (e.g., the captured images captured by each of left sidecamera 3 a, right side camera 3 b, front camera 3 c, and rear camera 3 dand the subject-vehicle image) and outputs the overhead image to displayapparatus 5. The overhead image is thus displayed on display apparatus5. Processing of generating the overhead image performed by generator 17will be described in detail later. Besides, the first to third overheadimages to be described later, which are generated and output bygenerator 17 to display apparatus 5, correspond to an example of a“display overhead image.”

Display control apparatus 200 has been described above.

Next, an operation of display control apparatus 200 will be describedwith reference to FIG. 3 . FIG. 3 is a flowchart illustrating anexemplary operation of display control apparatus 200.

A flow of FIG. 3 starts at the beginning of a search for a vacantparking space. Incidentally, as described above, the respective kinds ofprocessing by each camera, subject-vehicle position sensor 4, anddetector 11 are repeatedly performed during the search for the vacantparking space.

First, determiner 16 determines whether a vacant parking space isdetected by detector 11 (step S1).

In a case where a vacant parking space is not detected by detector 11(step S1: NO), the flow returns to step S1.

On the other hand, in a case where the vacant parking space is detectedby detector 11 (step S1: YES), receiver 14 receives the locationalinformation on the vacant parking space detected by detector 11 and thecaptured image of the vacant parking space captured by at least one ofeach camera to store them in storage 15 in association with each other(step S2). The processing of step S2 is performed for each detection ofthe vacant parking space.

Next, generator 17 reads the notification image from storage 15 andoutputs the notification image to display apparatus 5 (step S3).

Thus, display apparatus 5 displays the notification image. Accordingly,the occupant of vehicle 1 can recognize that the vacant parking space isdetected. Note that, the process of step S3 may be performed for eachdetection of the vacant parking space.

Next, determiner 16 determines whether vehicle 1 is stopped (step S4).

Here, in a case where vehicle 1 is not stopped (step S4: NO), the flowreturns to step S1.

On the other hand, when vehicle 1 is stopped (step S4: YES), determiner16 determines whether all the vacant spaces detected by detector 11 areindicated in an overhead image to be generated (step S5).

For example, determiner 16 determines whether all the vacant spacesdetected are indicated in the overhead image to be generated on thebasis of the locational information on all the vacant parking spacesread from storage 15, subject-vehicle positional information receivedfrom subject-vehicle position sensor 4 (here, information indicating astopping position of vehicle 1), and the viewing angle of each camera.

In a case where all the vacant parking spaces are indicated in theoverhead image (step S5: YES), the flow proceeds to steps S6 and S7. Onthe other hand, in a case where not all the vacant parking space areindicated in the overhead image (step S5: NO), the flow proceeds tosteps S8 and S9.

First, steps S6 and S7 will be described.

Generator 17 generates the first overhead image based on the imagescaptured by each camera in a current position (e.g., the stoppingposition) of vehicle 1 (hereinafter collectively referred to as a“current position captured image”) and the subject-vehicle image (stepS6).

The first overhead image is a conventional and common overhead image.Here, for step S6, a description will be given with an example in whichone vacant parking space is detected, and a captured image of the vacantparking space is included in the current position captured image.

First, generator 17 performs, for example, distortion correctionprocessing or projection conversion processing on the current positioncaptured image read from storage 15 (including the captured image of thedetected vacant parking space). Generator 17 then generates the firstoverhead image by combining the current position overhead image(including the overhead image of the detected vacant parking space)obtained by the processing and the subject-vehicle image read fromstorage 15. During this generation, generator 17 performs an arrangementof the subject-vehicle image and the overhead image of the vacantparking space based on the subject-vehicle positional informationreceived from subject-vehicle position sensor 4 (here, the informationindicating the stopping position of vehicle 1) and the locationalinformation of the detected vacant parking space.

Next, generator 17 outputs the first overhead image generated in step S6to display apparatus 5 (step S7).

Thus, display apparatus 5 displays the first overhead image generated instep S6.

Next, steps S8 to S9 will be described.

Generator 17 generates the second overhead image based on the currentposition captured image, the subject-vehicle image, and the imagescaptured by each camera at a position where vehicle 1 has beenpositioned (e.g., a position during traveling) before the current pointin time (e.g., at the time of stopping) (hereinafter collectivelyreferred to as a “past position captured image”) (step S8).

Here, for step S8, a description will be given with an example in whichthe detected vacant parking spaces are two parking spaces BS1 and BS2illustrated in FIG. 1 , a captured image of parking space BS2 isincluded in the current position captured image, and a captured image ofparking space BS1 is included in the past position captured image (e.g.,parking space BS1 is outside the viewing angle of each camera in thecurrent position of vehicle 1).

First, generator 17 performs, for example, distortion correctionprocessing or projection conversion processing on the current positioncaptured image read from storage 15 (including the captured image ofparking space BS2 which is detected). Generator 17 then generates thefirst overhead image by combining the current position overhead imageobtained by the processing (including the overhead image of parkingspace BS2 which is detected) and the subject-vehicle image read fromstorage 15. In this generation, generator 17 performs arrangement of thesubject-vehicle image and the overhead image of parking space BS2 basedon the subject-vehicle positional information received fromsubject-vehicle position sensor 4 (here, the information indicating thestopping position of vehicle 1) and the positional information ofparking space BS2.

Next, generator 17 performs, for example, distortion correctionprocessing or projection conversion processing on the past positioncaptured image (including the captured image of parking space BS1 whichis detected) read from storage 15 and thereby obtains a past positionoverhead image (including the overhead image of parking space BS1 whichis detected).

Generator 17 then generates the second overhead image by combining thepast position overhead image (including the overhead image of parkingspace BS1) with respect to the first overhead image including thesubject-vehicle image and the current position overhead image (includingthe overhead image of parking space BS2). During this generation,generator 17 performs an arrangement of the overhead image of parkingspace BS1 based on the locational information on parking space BS1 readfrom storage 15. At this time, the respective images in the secondoverhead image may be adjusted (reduced) in size depending on a displayrange of an image on display apparatus 5.

The second overhead image generated in this manner includes, unlike thefirst overhead image, the overhead image of the vacant parking spacegenerated based on the captured image of the vacant parking spacecaptured at the position where vehicle 1 has been positioned before thecurrent point in time. That is, the second overhead image includes anoverhead image generated from the image captured at the position wherevehicle 1 has been positioned before the position where the image usedfor generating the first overhead image is captured.

Step S8 has been described above, but the processing order in step S8 isnot limited to the above description.

Next, generator 17 outputs the second overhead image generated in stepS8 to display apparatus 5 (step S9).

Thus, display apparatus 5 displays the second overhead image generatedin step S8. FIG. 4 illustrates an example of the second overhead imagedisplayed on apparatus 5.

As illustrated in FIG. 4 , the second overhead image includessubject-vehicle image I1 indicating vehicle 1, parking space image I2indicating parking space BS2 which is the detected vacant parking space,and parking space image I3 indicating parking space BS1 which is thedetected vacant parking space.

Incidentally, step S8 has been described above with an example with anexample in which the parking space outside the viewing angle of eachcamera in the current position of vehicle 1 is only parking space BS1among the detected vacant parking spaces, but the present disclosure isnot limited to this. In a case where there is a plurality of parkingspaces outside the viewing angle of each camera in the current positionof vehicle 1 among the detected vacant parking spaces, the secondoverhead image is generated to include overhead images of these parkingspaces.

Furthermore, step S8 has been described above with an example with anexample in which the current position captured image includes only thecaptured image of parking space BS2 which is detected, but the presentdisclosure is not limited to this. In a case where another vehicle parksin parking space B S2 and a captured image of the other vehicle isincluded in the current position captured image, the second overheadimage illustrated in FIG. 4 is assumed to include only parking spaceimage I3 and not to include parking space image I2.

The operation of display control apparatus 200 has been described above.

As described in detail above, according to the present embodiment, evenwhen the detected vacant parking space is outside a current viewingangle of each camera, vehicle 1 generates the overhead image of thevacant parking space based on the image of the vacant parking spacecaptured at the position where vehicle 1 has been positioned before thecurrent point in time, generates then the second overhead imageincluding the overhead image of the vacant parking space, and displaysthe second overhead image. Consequently, the occupant of vehicle 1 caneasily recognize a location of the detected vacant parking space.

The embodiment described above can be variously modified withoutdeparting from the spirit thereof. Hereinafter, variations will bedescribed.

[Variation 1]

In the above embodiment, it has been described that in a case where thedetected vacant parking space is outside the viewing angle of eachcamera in the current position of vehicle 1, there arises the problem inthat the occupant of vehicle 1 cannot recognize the location of thevacant parking space. Similarly, the same problem arises in a case wherethe detected vacant parking space is hidden behind a static objectadjacent thereto as seen from the current position of vehicle 1. Anobject of the present variation is to solve the problem.

An operation of display control apparatus 200 according to the presentvariation will be described with reference to FIG. 5 . FIG. 5 is aflowchart illustrating an exemplary operation of display controlapparatus 200 according to the present variation. FIG. 5 is a flowchartin which steps S11 to 15 are added to FIG. 3 . In the following, stepsS11 to S15 will be mainly described.

A flow of FIG. 5 starts at the beginning of a search for a vacantparking space. Note that, the respective kinds of processing by eachcamera, subject-vehicle position sensor 4, and detector 11 arerepeatedly performed during the search for the vacant parking space.

First, determiner 16 determines whether a static object is detected bydetector 11 (step S11).

In a case where a static object is detected by detector 11 (step S11:YES), receiver 14 receives positional information on the static objectdetected by detector 11 and a captured image of the static objectcaptured by at least one of each camera to store them in storage 15 inassociation with each other (step S12). The processing of step S12 isperformed for each detection of the static object.

Incidentally, in a case where the static object is not detected bydetector 11 (step S11: NO), the flow proceeds to step S1.

Steps S1 to S7 have already been described with reference to FIG. 3 inthe above embodiment, and thus description thereof will be omitted.

In this variation, in a case where not all the vacant spaces areindicated in an overhead image to be generated (step S5: NO), the flowproceeds to step S13.

Determiner 16 determines whether there is a vacant parking space hiddenbehind the static object as seen from the current position of vehicle 1among the detected vacant parking spaces (step S13).

For example, determiner 16 determines, for each detected vacant parkingspace, whether the vacant parking space is hidden behind the staticobject on the basis of the subject-vehicle position information, thelocational information on the vacant parking space, and positionalinformation on the static object.

Here, a specific example of the case where the detected vacant parkingspace is hidden behind the static object will be described withreference to FIG. 1 .

In FIG. 1 , for example, in a situation where parking space BS1 isdetected as a vacant parking space and vehicle 1 parks in front ofparking space BS2, parking space BS1 is hidden behind parked vehicle V2as seen from the position of vehicle 1. In this case, determiner 16determines that parking space BS1 is hidden behind parked vehicle V2based on the subject-vehicle positional information, the locationalinformation on parking space BS1, and the positional information onparked vehicle V2.

Hereinafter, the description returns to FIG. 5 .

In a case where there is no vacant parking space hidden behind thestatic object among the detected vacant parking spaces (step S13: NO),the flow proceeds to steps S8 and S9.

Steps S8 and S9 have already been described with reference to FIG. 3 inthe above embodiment, and thus description thereof will be omitted.

In a case where there is a vacant parking space hidden behind the staticobject among the detected vacant parking spaces (step S13: YES), theflow proceeds to step S14.

Generator 17 generates the third overhead image based on the currentposition captured image, the past position captured image, and thesubject-vehicle image (step S14).

Here, for step S14, a description will be given with an example in whichthe detected vacant parking spaces are two parking spaces BS1 and BS2illustrated in FIG. 1 , a captured image of parking space BS2 isincluded in the current position captured image, and the respectivecaptured images of parked vehicles V1 and V2, and parking space BS1 areincluded in the past position captured image (e.g., parked vehicles V1and V2, and parking space BS1 are outside the viewing angle of eachcamera in the current position of vehicle 1).

In addition, the respective captured images of parked vehicles V1 and V2are assumed to be, for example, images captured when left side camera 3a is positioned at a center of a vehicle width direction of each ofparked vehicles V1 and V2.

First, generator 17 performs, for example, distortion correctionprocessing or projection conversion processing on the current positioncaptured image read from storage 15 (including the captured image ofparking space BS2 which is detected). Generator 17 then generates thefirst overhead image by combining the current position overhead imageobtained by the processing (including the overhead image of parkingspace BS2 which is detected) and the subject-vehicle image read fromstorage 15. In this generation, generator 17 performs arrangement of thesubject-vehicle image and the overhead image of parking space BS2 basedon the subject-vehicle positional information received fromsubject-vehicle position sensor 4 (here, the information indicating thestopping position of vehicle 1) and the positional information ofparking space BS2.

Next, generator 17 performs, for example, distortion correctionprocessing or projection conversion processing on the current positioncaptured image read from storage 15 (including the respective capturedimages of parking space BS1 and parked vehicles V1 and V2 which aredetected). Generator 17 thereby obtains a past position overhead image(including the respective overhead images of parking space BS1 andparked vehicles V1 and V2 which are detected).

Generator 17 then generates the third overhead image by combining thepast position overhead image (including the respective overhead imagesof parking space BS1 and parked vehicles V1 and V2 which are detected)with respect to the first overhead image including the subject-vehicleimage and the current position overhead image (including the overheadimage of parking space BS2). During this generation, generator 17performs an arrangement of the overhead image of parking space BS1 andparked vehicles V1 and V2 based on the positional information on each ofparking space BS1 and parked vehicles V1 and V2 read from storage 15. Atthis time, the respective images in the third overhead image may beadjusted (reduced) in size depending on a display range of an image ondisplay apparatus 5.

Step S13 has been described above, but the processing order in step S13is not limited to the above description.

Next, generator 17 outputs the third overhead image generated in stepS13 to display apparatus 5 (step S15).

Thus, display apparatus 5 displays the third overhead image generated instep S13. FIG. 6 illustrates an example of the third overhead imagedisplayed on display apparatus 5.

As illustrated in FIG. 6 , the third overhead image includes parkedvehicle image I4 indicating parked vehicle V1 and parked vehicle imageI5 indicating parked vehicle V2, in addition to subject-vehicle imageI1, parking space image I2, and parking space image I3 which areillustrated in FIG. 4 .

Note that, for simplicity of description, FIG. 6 illustrates anexemplary case where parked vehicle images I4 and I5 clearly illustrateshapes of parked vehicles V1 and V2 respectively, but the presentdisclosure is not limited to this. Generally, when a captured imageincluding a plane (e.g., parking space) is converted into an overheadimage, the shape of the plane in the overhead image becomes a shape asseen from directly above, whereas, when a captured image including athree-dimensional object (e.g., parked vehicle) is converted into anoverhead image, the shape of the three-dimensional object in theoverhead image becomes a shape that differs from the shape as seen fromdirectly above. For example, in a case where the captured imageincluding a parked vehicle is converted into an overhead image, theshape of the parked vehicle in the overhead image is to be distorted(e.g., shape such as obliquely upward protrusion). Consequently, therespective shapes of parked vehicles V1 and V2 in parked vehicle imagesI4 and I5 may be distorted.

Furthermore, for example, in a case where parked vehicle image I5overlaps with parking space image I3 due to a large distortion of theshape of parked vehicle V2 in parked vehicle image I5 (e.g., protrusionin a direction of the overhead image of parking space BS1), generator 17may perform processing such as deleting an overlapped part with parkingspace image I3 in parked vehicle image I5 when generating the thirdoverhead image. Thus, the third overhead image displayed on displayapparatus 5 includes parked vehicle image I5 from which the overlappedpart with parking space image I3 has been deleted.

Further, in the present variation, a description has been given with anexample in which the overhead image generated based on the capturedimage of the static object (e.g., parked vehicle images I4 and I5), butthe preset disclosure is not limited to this. A prepared parked vehicleimage may be used. This parked vehicle image is, like subject-vehicleimage I1, an overhead image of a predetermined vehicle.

The operation of display control apparatus 200 according to the presentvariation has been described above.

According to the present variation, in a case where the detected vacantparking space is hidden behind the static object as seen from thecurrent position of vehicle 1, an image indicating the static object isalso displayed in addition to the image indicating all the detectedvacant parking spaces and the subject-vehicle image; thus, the occupantof vehicle 1 can recognize more easily the location of the detectedvacant parking space.

[Variation 2]

In the embodiment and Variation 1, a description has been given with anexample in which the notification image is output to display apparatus 5when a vacant parking space is detected; however, output processing ofthe notification image may not be performed.

Moreover, in the embodiment and Variation 1, a description has beengiven with an example in which the determination processing of step S5is performed with the stopping of vehicle 1 as a trigger after thevacant parking space is detected, but the present disclosure is notlimited to this. For example, the determination processing of step S5may be performed with an operation instructing the displaying of theoverhead image by the occupant of vehicle 1 as a trigger.

Furthermore, the second overhead image or third overhead image may becontinuously generated and displayed while vehicle 1 is traveling forsearching for a vacant parking space. For example, the second overheadimage may be updated so that an overhead image of a vacant parking spacepresent within a predetermined range from the current position ofvehicle 1 is displayed while vehicle 1 is traveling. In addition, forexample, the third overhead image may be updated so that overhead imagesof a vacant parking space and a static object, which are present withina predetermined range from the current position of vehicle 1, aredisplayed while vehicle 1 is traveling.

[Variation 3]

FIG. 2 illustrates an exemplary case where display control apparatus 200is included in parking assistance apparatus 100, but display controlapparatus 200 and parking assistance apparatus 100 may be separate fromeach other. In this case, display control apparatus 200 receivesinformation indicating processing results by detector 11 and determiner16 from parking assistance apparatus 100 and performs the respectivekinds of processing described above based on the information (e.g., theprocessing performed by each of determiner 16 and generator 17)

[Variation 4]

In the embodiment, a description has been given with an example in whichdetector 11 detects a vacant parking space or a static object based onthe image captured by the vehicle-mounted camera (e.g., camera group 3),but the present disclosure is not limited to this. For example, in acase where vehicle 1 is equipped with a detecting device other than avehicle-mounted camera, the vacant parking space or static object may bedetected based on signals received from the detecting device. Examplesof detecting devices other than the vehicle-mounted camera include, forexample, a radar device that transmits radio waves to the periphery ofvehicle 1 and receives the reflected waves or an ultrasound sonar thattransmits sound waves to the periphery of vehicle 1 and receives thereflected waves. As the radar device, for example, a millimeter waveradar or laser radar may be used. The laser radar is also referred to asLight Detection and Ranging (LIDAR).

In order to further improve the detection accuracy, detector 11 maydetect a vacant parking space or a static object based on both of imagescaptured by the vehicle-mounted camera and signals indicating reflectedwaves received by the detecting device.

[Variation 5]

In the embodiment, a description has been given with an example in whichvehicle 1 is a vehicle capable of performing automatic parking, butvehicle 1 may be a vehicle not capable of performing automatic parking.In this case, traveling controller 12 controls actuator group 6 based onoperations for acceleration and deceleration, braking and steering by anoccupant of vehicle 1.

Variations have been described above. Incidentally, the respectivevariations may be realized in combination as appropriate.

(Summary of the Present Disclosure)

The summary of the present disclosure is as follows.

A display control apparatus of the present disclosure includes: areceiver that, for each detection of a vacant parking space present in aperiphery of a vehicle, receives locational information on the vacantparking space and a captured image of the vacant parking space capturedby a camera mounted on the vehicle and stores, in a storage, thelocational information on the vacant parking space and the capturedimage of the vacant parking space in association with each other; and agenerator that generates an overhead image of the vacant parking spacebased on a captured image of the vacant parking space captured at aposition where the vehicle has been positioned before a current point intime, generates a display overhead image including at least the overheadimage of the vacant parking space and the overhead image of the vehiclebased on positional information on each of the vacant parking space andthe vehicle, and outputs the display overhead image to a displayapparatus mounted on the vehicle, wherein the display overhead image tobe displayed on the display apparatus includes at least one of anoverhead image of the vacant parking space outside of a current viewingangle of the camera and an overhead image of the vacant parking spacehidden behind a static object as seen from a current position of thevehicle.

In the display control apparatus of the present disclosure, the displayoverhead image is updated so that an overhead image of the vacantparking space present within a predetermined range from the currentposition of the vehicle is displayed while the vehicle is traveling.

In the display control apparatus of the present disclosure, thereceiver, for each detection of a static object present in a peripheryof the vehicle, receives positional information on the static object anda captured image of the static object captured by the camera and stores,in the storage, the positional information on the static object and thecaptured image of the static object in association with each other; andthe generator generates an overhead image of the static object based onthe captured image of the static object and outputs the overhead imageof the static object to the display apparatus by further including theoverhead image of the static object in the display overhead image basedon the positional information on the static object.

In the display control apparatus of the present disclosure, the displayoverhead image to be displayed on the display apparatus includes anoverhead image of the static object outside a viewing angle of thecamera.

In the display control apparatus of the present disclosure, the displayoverhead image to be displayed on the display apparatus includes anoverhead image of the static object from which an overlapped part withthe overhead image of the vacant parking space has been deleted.

In the display control apparatus of the present disclosure, the displayoverhead image is updated so that overhead images respectively of thevacant parking space and the static object which are present within apredetermined range from the current position of the vehicle aredisplayed while the vehicle is traveling.

In the display control apparatus of the present disclosure, the vehicleis a vehicle that performs automatic parking in the vacant parkingspace.

A parking assistance apparatus of the present disclosure includes: adetector that detects a vacant parking space present in a periphery of avehicle; a traveling controller that controls traveling of the vehicleand automatically parks the vehicle in the vacant parking space; areceiver that, for each detection of the vacant parking space by thedetector, receives locational information on the vacant parking spaceand a captured image of the vacant parking space captured by a cameramounted on the vehicle and stores the locational information on thevacant parking space and the captured image of the vacant parking spacein association with each other; and a generator that generates anoverhead image of the vacant parking space based on a captured image ofthe vacant parking space captured at a position where the vehicle hasbeen positioned before a current point in time, generates a displayoverhead image including at least the overhead image of the vacantparking space and the overhead image of the vehicle based on positionalinformation on each of the vacant parking space and the vehicle, andoutputs the display overhead image to a display apparatus mounted on thevehicle, wherein the display overhead image to be displayed on thedisplay apparatus includes at least one of an overhead image of thevacant parking space outside of a current viewing angle of the cameraand an overhead image of the vacant parking space hidden behind a staticobject as seen from a current position of the vehicle.

A display control method of the present disclosure is performed by anapparatus mounted on a vehicle, the display control method including:receiving, for each detection of a vacant parking space present in aperiphery of the vehicle, locational information on the vacant parkingspace and a captured image of the vacant parking space captured by acamera mounted on the vehicle and storing the locational information onthe vacant parking space and the captured image of the vacant parkingspace in association with each other; and generating an overhead imageof the vacant parking space based on a captured image of the vacantparking space captured at a position where the vehicle has beenpositioned before a current point in time, generating a display overheadimage including at least the overhead image of the vacant parking spaceand the overhead image of the vehicle based on positional information oneach of the vacant parking space and the vehicle, and outputting thedisplay overhead image to a display apparatus mounted on the vehicle,wherein the display overhead image to be displayed on the displayapparatus includes at least one of an overhead image of the vacantparking space outside of a current viewing angle of the camera and anoverhead image of the vacant parking space hidden behind a static objectas seen from a current position of the vehicle.

While various embodiments have been described herein above, it is to beappreciated that various changes in form and detail may be made withoutdeparting from the spirit and scope of the invention(s) presently orhereafter claimed.

This application is entitled to and claims the benefit of JapanesePatent Application No. 2019-048278, filed on Mar. 15, 2019, thedisclosure of which including the specification, drawings and abstractis incorporated herein by reference in its entirety.

INDUSTRIAL APPLICABILITY

A display control apparatus, a parking assistance apparatus and adisplay control method of the present disclosure is useful for assistingvehicles with parking.

1. A display control apparatus, comprising: a receiver that, for eachdetection of a vacant parking space present in a periphery of a vehicle,receives locational information on the vacant parking space and acaptured image of the vacant parking space captured by a camera mountedon the vehicle and stores, in a storage, the locational information onthe vacant parking space and the captured image of the vacant parkingspace in association with each other; and a generator that generates anoverhead image of the vacant parking space based on a captured image ofthe vacant parking space captured at a position where the vehicle hasbeen positioned before a current point in time, generates a displayoverhead image including at least the overhead image of the vacantparking space and the overhead image of the vehicle based on positionalinformation on each of the vacant parking space and the vehicle, andoutputs the display overhead image to a display apparatus mounted on thevehicle, wherein the display overhead image to be displayed on thedisplay apparatus includes at least one of an overhead image of thevacant parking space outside of a current viewing angle of the cameraand an overhead image of the vacant parking space hidden behind a staticobject as seen from a current position of the vehicle.
 2. The displaycontrol apparatus according to claim 1, wherein the display overheadimage is updated so that an overhead image of the vacant parking spacepresent within a predetermined range from the current position of thevehicle is displayed while the vehicle is traveling.
 3. The displaycontrol apparatus according to claim 1, wherein the receiver, for eachdetection of a static object present in a periphery of the vehicle,receives positional information on the static object and a capturedimage of the static object captured by the camera and stores, in thestorage, the positional information on the static object and thecaptured image of the static object in association with each other; andthe generator generates an overhead image of the static object based onthe captured image of the static object and outputs the overhead imageof the static object to the display apparatus by further including theoverhead image of the static object in the display overhead image basedon the positional information on the static object.
 4. The displaycontrol apparatus according to claim 3, wherein the display overheadimage to be displayed on the display apparatus includes an overheadimage of the static object outside a viewing angle of the camera.
 5. Thedisplay control apparatus according to claim 3, wherein the displayoverhead image to be displayed on the display apparatus includes anoverhead image of the static object from which an overlapped part withthe overhead image of the vacant parking space has been deleted.
 6. Thedisplay control apparatus according to claim 3, wherein the displayoverhead image is updated so that overhead images respectively of thevacant parking space and the static object which are present within apredetermined range from the current position of the vehicle aredisplayed while the vehicle is traveling.
 7. The display controlapparatus according to claim 1, wherein the vehicle is a vehicle thatperforms automatic parking in the vacant parking space.
 8. The displaycontrol apparatus according to claim 4, wherein the display overheadimage to be displayed on the display apparatus includes an overheadimage of the static object from which an overlapped part with theoverhead image of the vacant parking space has been deleted.
 9. Thedisplay control apparatus according to claim 4, wherein the displayoverhead image is updated so that overhead images respectively of thevacant parking space and the static object which are present within apredetermined range from the current position of the vehicle aredisplayed while the vehicle is traveling.
 10. The display controlapparatus according to claim 5, wherein the display overhead image isupdated so that overhead images respectively of the vacant parking spaceand the static object which are present within the predetermined rangefrom the current position of the vehicle are displayed while the vehicleis traveling.
 11. The display control apparatus according to claim 2,wherein the vehicle is a vehicle that performs automatic parking in thevacant parking space.
 12. The display control apparatus according toclaim 3, wherein the vehicle is a vehicle that performs automaticparking in the vacant parking space.
 13. The display control apparatusaccording to claim 4, wherein the vehicle is a vehicle that performsautomatic parking in the vacant parking space.
 14. The display controlapparatus according to claim 5, wherein the vehicle is a vehicle thatperforms automatic parking in the vacant parking space.
 15. The displaycontrol apparatus according to claim 6, wherein the vehicle is a vehiclethat performs automatic parking in the vacant parking space.
 16. Thedisplay control apparatus according to claim 8, wherein the vehicle is avehicle that performs automatic parking in the vacant parking space. 17.The display control apparatus according to claim 9, wherein the vehicleis a vehicle that performs automatic parking in the vacant parkingspace.
 18. The display control apparatus according to claim 10, whereinthe vehicle is a vehicle that performs automatic parking in the vacantparking space.
 19. A parking assistance apparatus, comprising: adetector that detects a vacant parking space present in a periphery of avehicle; a traveling controller that controls traveling of the vehicleand automatically parks the vehicle in the vacant parking space; areceiver that, for each detection of the vacant parking space by thedetector, receives locational information on the vacant parking spaceand a captured image of the vacant parking space captured by a cameramounted on the vehicle and stores the locational information on thevacant parking space and the captured image of the vacant parking spacein association with each other; and a generator that generates anoverhead image of the vacant parking space based on a captured image ofthe vacant parking space captured at a position where the vehicle hasbeen positioned before a current point in time, generates a displayoverhead image including at least the overhead image of the vacantparking space and the overhead image of the vehicle based on positionalinformation on each of the vacant parking space and the vehicle, andoutputs the display overhead image to a display apparatus mounted on thevehicle, wherein the display overhead image to be displayed on thedisplay apparatus includes at least one of an overhead image of thevacant parking space outside of a current viewing angle of the cameraand an overhead image of the vacant parking space hidden behind a staticobject as seen from a current position of the vehicle.
 20. A displaycontrol method performed by an apparatus mounted on a vehicle, thedisplay control method comprising: receiving, for each detection of avacant parking space present in a periphery of the vehicle, locationalinformation on the vacant parking space and a captured image of thevacant parking space captured by a camera mounted on the vehicle andstoring the locational information on the vacant parking space and thecaptured image of the vacant parking space in association with eachother; and generating an overhead image of the vacant parking spacebased on a captured image of the vacant parking space captured at aposition where the vehicle has been positioned before a current point intime, generating a display overhead image including at least theoverhead image of the vacant parking space and the overhead image of thevehicle based on positional information on each of the vacant parkingspace and the vehicle, and outputting the display overhead image to adisplay apparatus mounted on the vehicle, wherein the display overheadimage to be displayed on the display apparatus includes at least one ofan overhead image of the vacant parking space outside of a currentviewing angle of the camera and an overhead image of the vacant parkingspace hidden behind a static object as seen from a current position ofthe vehicle.